Control device, control method, and storage medium

ABSTRACT

A control device includes circuitry configured to: generate a bird’s-eye view image and a three-dimensional image that show a moving body and surroundings of the moving body, based on respective pieces of imaging data obtained by a plurality of imaging devices of the moving body: cause a display device to display the generated bird’s-eye view image and the generated three-dimensional image; and determine whether a predetermined object is present in a boundary region between the respective pieces of imaging data in the bird’s-eye view image and the three-dimensional image. Upon determining that the predetermined object is present in the boundary region, the circuitry is configured to preferentially change the boundary region in the three-dimensional image among the displayed bird’s-eye view image and the displayed three-dimensional image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 fromJapanese Patent Application No. 2021-190067 filed on Nov. 24, 2021, thecontents of which are incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to a control device, a control method,and a storage medium storing a control program.

BACKGROUND ART

In recent years, as a specific measure against global climate change,efforts for implementing a low-carbon society or a decarbonized societyhave become active. Also in vehicles, reduction in CO₂ emission isstrongly required, and automatic driving of vehicles and introduction ofdriving assistance that contribute to improvement in fuel efficiency arerapidly progressing.

In the related art, an image generation method has been known in which apredetermined range is imaged by each of cameras mounted on front rear,left, and right sides of a vehicle, a surroundings image (for example, abird’s-eye view image) of the vehicle and the surroundings of thevehicle is generated based on a combined image of the captured images,and a three-dimensional image is generated based on the bird’s-eye viewimage. Japanese Patent Publication No. 51 12998 (hereinafter, referredto as Patent Literature 1) discloses a vehicle surroundings monitoringdevice that changes an imaging range of each camera in accordance withopening and closing of a side mirror of a vehicle, and that changes aboundary position between captured images in a combined image of thecaptured images to generate a bird’s-eye view image. Further, JapanesePatent Application Laid-Open Publication No. 2013-093865 (hereinafter,referred to as Patent Literature 2) discloses a vehicle surroundingsmonitoring device in which a boundary line on a generated bird’s-eyeview image is changed with respect to a target whose entirety is notdisplayed on the generated bird’s-eye view image, and the entirety ofthe target is displayed.

For example, a bird’s-eye view image or a three-dimensional image may bedisplayed on a display device of a vehicle at the time of parkingassistance of the vehicle. At this time, boundary lines between capturedimages in the bird’s-eye view image or the three-dimensional image maybe displayed overlapping a parking frame the vehicle is to be parked inor a parking frame the vehicle is in the process of being parked in. Inthis case, an image of the parking frame in which the boundary lines aredisplayed overlapping the parking frame becomes a distorted image, has alowered visibility, and thus is not desirable as an image at the time ofparking.

However, Patent Literature 1 and Patent Literature 2 do not describe acorrespondence relationship between a parking frame and a boundary linebetween captured images at the time of parking assistance. Therefore,there is room for improvement in the visibility of the bird’s-eye viewimage and the three-dimensional image at the time of parking assistance.

An object of the present disclosure is to provide a control device, acontrol method, and a storage medium storing a control program capableof displaying a surroundings image of a moving body that enables quickrecognizing of a predetermined object.

SUMMARY

A first aspect of the present disclosure relates to a control device,including:

-   circuitry configured to:-   generate a bird’s-eye view image and a three-dimensional image that    show a moving body and surroundings of the moving body, based on    respective pieces of imaging data obtained by a plurality of imaging    devices of the moving body;-   cause a display device to display the generated bird’s-eye view    image and the generated three-dimensional image; and-   determine whether a predetermined object is present in a boundary    region between the respective pieces of imaging data in the    bird’s-eye view image and the three-dimensional image, in which-   upon determining that the predetermined object is present in the    boundary region, the circuitry is configured to preferentially    change the boundary region in the three-dimensional image among the    displayed bird’s-eye view image and the displayed three-dimensional    image.

A second aspect of the present disclosure relates to a control methodexecuted by a processor, in which

-   the processor is configured to generate a bird’s-eye view image and    a three-dimensional image that show a moving body and surroundings    of the moving body based on respective pieces of imaging data    obtained by a plurality of imaging devices of the moving body, and    display the generated bird’s-eye view image and the generated    three-dimensional image on a display device, and-   the control method includes:-   the processor determining whether a predetermined object is present    in a boundary region between the respective pieces of imaging data    in the bird’s-eye view image and the three-dimensional image; and-   upon determining that the predetermined object is present in the    boundary region, the processor preferentially changing the boundary    region in the three-dimensional image among the displayed bird’s-eye    view image and the displayed three-dimensional image

A third aspect of the present disclosure relates to a non-transitorycomputer-readable storage medium storing a control program for causing aprocessor to perform processing, in which

-   the processor is configured to generate a bird’s-eye view image and    a three-dimensional image that show a moving body and surroundings    of the moving body based on respective pieces of imaging data    obtained by a plurality of imaging devices of the moving body, and    to display the generated bird’s-eye view image and the generated    three-dimensional image on a display device, and-   the processing includes:-   determining whether a predetermined object is present in a boundary    region between the respective pieces of imaging data in the    bird’s-eye view image and the three-dimensional image; and-   when it is determined that the predetermined object is present in    the boundary region, preferentially changing the boundary region in    the three-dimensional image among the displayed bird’s-eye view    image and the displayed three-dimensional image.

According to the control device, the control method, and the controlprogram of the present disclosure, it is possible to display surroundingimages of a moving body that enables quick recognizing of apredetermined object.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a side view illustrating an example of a vehicle on which acontrol device of the present embodiment is mounted.

FIG. 2 is a top view of the vehicle illustrated in FIG. 1 .

FIG. 3 is a block diagram illustrating an internal configuration of thevehicle illustrated in FIG. 1 .

FIG. 4 is a diagram illustrating an example of a synthesized imagegenerated using respective pieces of imaging data of a plurality ofcameras.

FIG. 5 is a diagram illustrating a synthesized image obtained bychanging a boundary region of the synthesized image illustrated in FIG.4

FIG. 6 is a flowchart illustrating display control performed by acontrol ECU according to a first embodiment.

FIG. 7 is a diagram illustrating an example of a bird’s-eye view imagegenerated based on respective pieces of imaging data of a plurality ofcameras.

FIG. 8 is a diagram illustrating an example of a bird’s-eye view imageand a three-dimensional image displayed on a touch screen of a vehicle.

FIG. 9 is a flowchart illustrating display control performed by acontrol ECU according to a second embodiment.

FIG. 10 is a diagram illustrating an example of a bird’s-eye view imagegenerated based on respective pieces of imaging data of a plurality ofcameras according to a third embodiment.

FIG. 11 is a diagram illustrating an example of a three-dimensionalimage generated based on respective pieces of imaging data of aplurality of cameras.

FIG. 12 is a diagram illustrating an example of a bird’s-eye view imageand a three-dimensional image displayed on a touch screen of a vehicle.

DESCRIPTION OF EMBODIMENTS

Hereinafter, an embodiment of a control device, a control method, and astorage medium storing a control program according to the presentdisclosure will be described with reference to the accompanyingdrawings. Note that the drawings are to be viewed according toorientation of the reference signs. In the present specification and thelike, in order to simplify and clarify the description, a front-reardirection, a left-right direction, and an up-down direction aredescribed in accordance with directions viewed from a driver of avehicle 10 illustrated in FIGS. 1 and 2 . In the drawings, a front sideof the vehicle 10 is denoted by Fr, a rear side thereof is denoted byRr, a left side thereof is denoted by L, a right side thereof is denotedby R, an upper side thereof is denoted by U, and a lower side thereof isdenoted by D.

Vehicle 10 on Which Control Device of the Present Disclosure Is Mounted

FIG. 1 is a side view of the vehicle 10 on which a control deviceaccording to the present disclosure is mounted. FIG. 2 is a top view ofthe vehicle 10 illustrated in FIG. 1 . The vehicle 10 is an example of amoving body of the present disclosure.

The vehicle 10 is an automobile that includes a driving source (notillustrated) and wheels including drive wheels driven by power of thedriving source and steerable steering wheels. In the present embodiment,the vehicle 10 is a four-wheeled automobile having a pair of left andright front wheels and a pair of left and right rear wheels. The drivingsource of the vehicle 10 is, for example, an electric motor. The drivingsource of the vehicle 10 may be an internal combustion engine such as agasoline engine or a diesel engine, or may be a combination of anelectric motor and an internal combustion engine. The driving source ofthe vehicle 10 may drive the pair of left and right front wheels, thepair of left and right rear wheels, or four wheels of the pair of leftand right front wheels and the pair of left and right rear wheels. Boththe front wheels and the rear wheels may be steerable steering wheels,or the front wheels or the rear wheels may be steerable steering wheels.

The vehicle 10 further includes side mirrors 11L and 11R. The sidemirrors 11L and 11R are mirrors (rearview mirrors) that are provided atouter sides of front seat doors of the vehicle 10 and that allow adriver to check the rear side and rear lateral sides. Each of the sidemirrors 11L and 11R is fixed to a body of the vehicle 10 by a rotationshaft extending in the up-down direction, and can be opened and closedby rotating about the rotation shaft. The side mirrors 11L and 11R areelectrically opened and closed by, for example, a driver’s operation onan operation part provided in the vicinity of a driver’s seat of thevehicle 10. A width of the vehicle 10 in a state where the side mirrors11L and 11R are closed is narrower than the width of the vehicle 10 in astate where the side mirrors 11L and 11R are opened. Therefore, forexample, when the vehicle 10 enters a narrow parking space, the driveroften performs an operation of setting the side mirrors 11L and 11R tothe closed state so that the vehicle 10 does not collide with anobstacle in the surroundings of the vehicle 10.

The vehicle 10 further includes a front camera 12Fr, a rear camera 12Rr,a left lateral-side camera 12L, and a right lateral-side camera 12R. Thefront camera 12Fr is a digital camera that is provided in a frontportion of the vehicle 10 and images a front side of the vehicle 10. Therear camera 12Rr is a digital camera that is provided in a rear portionof the vehicle 10 and images a rear side of the vehicle 10. The leftlateral-side camera 12L, is a digital camera that is provided in theleft side mirror 11L of the vehicle 10 and images a left lateral side ofthe vehicle 10. The right lateral-side camera 12R is a digital camerathat is provided in the right side mirror 11R of the vehicle 10 andimages a right lateral side of the vehicle 10. The front camera 12Fr,the rear camera 12Rr, the left lateral-side camera 12L, and the rightlateral-side camera 12R are examples of an imaging device of the presentdisclosure.

Intemal Configuration of Vehicle 10

FIG. 3 is a block diagram illustrating an example of an internalconfiguration of the vehicle 10 illustrated in FIG. 1 . As illustratedin FIG. 3 , the vehicle 10 includes a sensor group 16, a navigationdevice 18, a control electronic control unit (ECU) 20. an electric powersteering (EPS) system 22, and a communication unit 24. The vehicle 10further includes a driving force control system 26 and a braking forcecontrol system 28. The control ECU 20 is an example of a control deviceof the present disclosure.

The sensor group 16 obtains various types of detection values used forcontrol performed by the control ECU 20. The sensor group 16 includesthe front camera 12Fr, the rear camera 12Rr, the left lateral-sidecamera 12L, and the right lateral-side camera 12R. In addition, thesensor group 16 includes a front sonar group 32 a, a rear sonar group 32b, a left lateral-side sonar group 32 c, and a right lateral-side sonargroup 32 d. Further, the sensor group 16 includes wheel sensors 34 a and34 b, a vehicle speed sensor 36, and an operation detector 38.

The front camera 12Fr, the rear camera 12Rr, the left lateral-sidecamera 12L, and the right lateral-side camera 12R output surroundingsimages obtained by imaging the surroundings of the vehicle 10. Thesurroundings images captured by the front camera 12Fr, the rear camera12Rr, the left lateral-side camera 12L, and the right lateral-sidecamera 12R are referred to as a front image, a rear image, a leftlateral-side image, and a right lateral-side image, respectively. Animage formed by the left lateral-side image and the right lateral-sideimage may be referred to as a lateral-side image.

The front sonar group 32 a, the rear sonar group 32 b, the leftlateral-side sonar group 32 c, and the right lateral-side sonar group 32d emit sound waves to the surroundings of the vehicle 10 and receivereflected sounds from other objects. The front sonar group 32 aincludes, for example, four sonars. The sonars constituting the frontsonar group 32 a are provided at an obliquely left front side, a frontleft side, a front right side, and an obliquely right front side of thevehicle 10, respectively. The rear sonar group 32 b includes, forexample, four sonars. The sonars constituting the rear sonar group 32 bare provided at an obliquely left rear side, a rear left side, a rearright side, and an obliquely right rear side of the vehicle 10,respectively. The left lateral-side sonar group 32 c includes, forexample, two sonars. The sonars constituting the left lateral-side sonargroup 32 c are provided at a front side and a rear side of a left sideportion of the vehicle 10, respectively. The right lateral-side sonargroup 32 d includes, for example, two sonars. The sonars constitutingthe right lateral-side sonar group 32 d are provided at a front side anda rear side of a right side portion of the vehicle 10, respectively.

The wheel sensors 34 a and 34 b detect a rotation angle of a wheel ofthe vehicle 10. The wheel sensors 34 a and 34 b may be implemented by anangle sensor or a displacement sensor. The wheel sensors 34 a and 34 boutput a detection pulse each time the wheel rotates by a predeterminedangle. The detection pulse output from the wheel sensors 34 a and 34 bis used to calculate the rotation angle of the wheel and a rotationspeed of the wheel. A movement distance of the vehicle 10 is calculatedbased on the rotation angle of the wheel. The wheel sensor 34 a detects,for example, a rotation angle θa of a left rear wheel. The wheel sensor34 b detects, for example, a rotation angle θb of a right rear wheel.

The vehicle speed sensor 36 detects a speed of a vehicle body of thevehicle 10, that is, a vehicle speed V. and outputs the detected vehiclespeed V to the control ECU 20. The vehicle speed sensor 36 detects thevehicle speed V based on, for example, rotation of a countershaft of thetransmission.

The operation detector 38 detects what operation is performed by a userusing an operation input part 14, and outputs the detected operation tothe control ECU 20. The operation input part 14 includes various userinterfaces such as a door mirror switch for switching between an openedstate and a closed state of the side mirrors 11L and 11R and a shiftlever (a select lever or a selector).

The navigation device 18 detects a current position of the vehicle 10using, for example, a global positioning system (GPS), and guides theuser to a route to a destination. The navigation device 18 includes astorage device (not illustrated) provided with a map informationdatabase.

The navigation device 18 includes a touch screen 42 and a speaker 44.The touch screen 42 functions as an input device and a display device ofthe control ECU 20. The user inputs various commands via the touchscreen 42. The touch screen 42 displays various screens. The user caninput, for example, a command related to parking assistance via thetouch screen 42. In addition, the touch screen 42 may display a screenrelated to parking assistance . For example, the touch screen 42displays a parking assistance button for requesting parking assistanceof the vehicle 10. The parking assistance button includes an automaticparking assistance button for requesting parking by automatic steeringof the control ECU 20 and a parking assistance button for requestingassistance at the time when parking is to be performed by an operationof the driver. Components other than the touch screen 42, for example, asmartphone may be used as the input device or the display device. Thespeaker 44 outputs various types of guidance information to an occupantof the vehicle 10 by voice.

The control ECU 20 includes an input/output unit 50, a calculator 52,and a storage unit 54. The calculator 52 is implemented by, for example,circuitry such as a central processing unit (CPU). The calculator 52performs various types of control by controlling units based on aprogram stored in the storage unit 54.

The calculator 52 includes a display controller 55, an objectpresence/absence determination unit 56, and an image processor 57. Theimage processor 57 generates a surroundings image of the vehicle 10based on imaging data obtained by the cameras of the vehicle 10.Specifically, the image processor 57 generates a synthesized image bysynthesizing respective pieces of imaging data obtained by the frontcamera 12Fr, the rear camera 12Rr, the left lateral-side camera 12L. andthe right lateral-side camera 12R, and generates a bird’s-eye view imageof the vehicle 10 and the surroundings of the vehicle 10 as viewed fromabove.

In addition, the image processor 57 performs image processing ofthree-dimensionally reconstructing the synthesized image of the piecesof imaging data obtained by the front camera 12Fr, the rear camera 12Rr,the left lateral-side camera 12L, and the right lateral-side camera 12R,and generates a three-dimensional image virtually showing a state inwhich the vehicle 10 and the surroundings of the vehicle 10 are rotatedand viewed from, for example, an obliquely upper side.

In addition, the image processor 57 sets a mask area in the generatedsurroundings image (the bird’s-eye view image and the three-dimensionalimage) . The mask area means an area set to hide the body of the vehicle10 reflected in a captured image of a camera. The mask area is set as anarea having a shape surrounding the vehicle 10. The image processor 57displays a vehicle image, which indicates the vehicle 10, in asuperimposed manner in a portion corresponding to a space in which thevehicle 10 is located in the mask area. The vehicle image is an imageshowing a state where the vehicle 10 is viewed from above, and isgenerated in advance and stored in the storage unit 54 or the like. Theimage processor 57 may set mask areas in the lateral-side images (theleft lateral-side image and the right lateral-side image) obtained bythe left lateral-side camera 12L and the right lateral-side camera 12R.

In addition, the image processor 57 performs re-synthesis processing onthe synthesized image of the imaging data obtained by the cameras tochange a boundary region between adjacent captured images in thesynthesized image. For example, when a predetermined object present in acaptured image is displayed overlapping a boundary region, the imageprocessor 57 changes the boundary region by performing re-synthesisprocessing according to a position of the predetermined target. Thepredetermined object is an object to be watched by the driver of thevehicle 10, such as a parking frame (parking space), a parking frameline, or an obstacle.

The object presence/absence determination unit 56 determines whether apredetermined object is present at a boundary region of the pieces ofimaging data in the bird’s-eye view image and the three-dimensionalimage generated by the image processor 57.

The display controller 55 causes the display device of the vehicle 10 todisplay the surroundings image generated by the image processor 57.Specifically, the display controller 55 causes the touch screen 42 todisplay the bird’s-eye view image and the three-dimensional image of thevehicle 10 generated by synthesizing the respective pieces of imagingdata of the front camera 12Fr, the rear camera 12Rr, the leftlateral-side camera 12L, and the right lateral-side camera 12R. Inaddition, the display controller 55 causes the touch screen 42 todisplay a bird’s-eye view image and a three-dimensional image on whichre-synthesis processing of the imaging data is performed based on adetermination result of the object presence/absence determination unit56.

Further, the control ECU 20 performs parking assistance of the vehicle10 by automatic steering in which an operation of a steering wheel 110is automatically performed under control of the control ECU 20. Theparking assistance is, for example, control of performing automaticsteering so as to stop in a parking frame selected by the driver of thevehicle 10 at the time of parking. In the assistance of automaticsteering, an accelerator pedal (not illustrated), a brake pedal (notillustrated), and the operation input part 14 are automaticallyoperated. In addition, when the user operates the accelerator pedal, thebrake pedal, and the operation input part 14 to park the vehicle 10, thecontrol ECU 20 performs auxiliary assistance.

The EPS system 22 includes a steering angle sensor 100, a torque sensor102, an EPS motor 104, a resolver 106, and an EPS ECU 108. The steeringangle sensor 100 detects a steering angle θstof the steering wheel 110.The torque sensor 102 detects a torque TQ applied to the steering wheel110.

The EPS motor 104 applies a driving force or a reaction force to asteering column 112 coupled to the steering wheel 110, thereby enablingoperation assistance of the steering wheel 110 and automatic steering atthe time of parking assistance for the driver. The resolver 106 detectsa rotation angle θm of the EPS motor 104. The EPS ECU 108 controls theentire EPS system 22. The EPS ECU 108 include an input/output unit (notillustrated), a calculator (not illustrated), and a storage unit (notillustrated), for example.

The communication unit 24 enables wireless communication with anothercommunication device 120. The other communication device 120 is a basestation, a communication device of another vehicle, an informationterminal such as a smartphone possessed by an occupant of the vehicle10, or the like.

The driving force control system 26 is provided with a driving ECU 130.The driving force control system 26 executes driving force control ofthe vehicle 10. The driving ECU 130 controls an engine or the like (notillustrated) based on an operation that the user performs on theaccelerator pedal (not illustrated), thereby controlling a driving forceof the vehicle 10.

The braking force control system 28 is provided with a braking ECU 132.The braking force control system 28 executes braking force control ofthe vehicle 10. The braking ECU 132 controls a braking force of thevehicle 10 by controlling a brake mechanism or the like (notillustrated) based on an operation that the user performs on the brakepedal (not illustrated), thereby controlling a braking force of thevehicle 10.

Change of Boundary Region by Image Processor 57

Next, processing of changing a boundary region in a synthesized image ofpieces of imaging data will be described with reference to FIGS. 4 and 5.

FIG. 4 is a diagram illustrating an example of a synthesized imagegenerated using respective pieces of imaging data obtained by the frontcamera 12Fr, the rear camera 12Rr, the left lateral-side camera 12L,andthe right lateral-side camera 12R. FIG. 5 is a diagram illustrating anexample of a synthesized image generated by changing a boundary regionof the synthesized image illustrated in FIG. 4 .

As illustrated in FIG. 4 , when generating a synthesized image 60, theimage processor 57 performs viewpoint conversion and correction in imagedistortion and the like on imaging data of imaging areas imaged by thefront camera 12Fr, the rear camera 12Rr, the left lateral-side camera12L. and the right lateral-side camera 12R so as to obtain imagespresenting the effect of viewing down from a predetermined viewpointposition vertically above the vehicle 10. Further, from converted imagesobtained by the conversion processing, the image processor 57 extracts afront image 61, a left lateral-side image 62, a right lateral-side image63. and a rear image 64 having predetermined view angle ranges that areset for respective converted images so that images on both sides of aboundary region match with each other. Then, the image processor 57synthesizes these images 61 to 64 to generate the synthesized image 60.A mask area 65 is provided in a central portion of the synthesized image60 so as to surround the vehicle 10. A vehicle image 67 indicating thevehicle 10 may be displayed in the mask area 65.

Boundary lines 66 a to 66 d, which are boundary regions of the capturedimages, are present between adjacent captured images of the front image61, the left lateral-side image 62, the right lateral-side image 63, andthe rear image 64. View angle ranges extracted from the front image 61,the left lateral-side image 62, the right lateral-side image 63, and therear image 64 may be field angle ranges by which boundary regionsbetween adjacent captured images match each other, and are not limitedto unique ranges. Therefore, positions of the boundary lines 66 a to 66d between the adjacent captured images can also be changed according tothe extracted view angle ranges.

View angle ranges of images that can be captured by the front camera12Fr, the rear camera 12Rr, the left lateral-side camera 12L, and theright lateral-side camera 12R are set to a size such that adjacentcaptured images overlap each other in an area of a certain extent.Therefore, a boundary region between adjacent captured images can bearbitrarily extracted from within the overlapping area, under acondition of matching of boundary region.

Boundary region images, particularly the images at the boundary lines 66a to 66 d, are extracted so that images on both sides of a boundaryregion match with each other. But since the boundary region images aresynthesis portions of different images, the visibility is often lowereddue to occurrence of distortion. Therefore, in a case where apredetermined object requiring good visibility is imaged to beoverlapped by the boundary lines 66 a to 66 d in the synthesized image60. the image processor 57 changes a position of a boundary region(boundary lines 66 a to 66 d) so that the boundary lines 66 a to 66 d donot overlap the object.

For example, it is assumed that the synthesized image 60 illustrated inFIG. 4 is a synthesized image indicating a state where a vehicle isabout to be parked in a certain parking space P. An obstacle 68 ispresent behind the vehicle in the parking space P. In this case, theobstacle 68 requires good visibility as a predetermined object. However,in the synthesized image 60, the obstacle 68 exists on the boundary line66 c between the left lateral-side image 62 and the rear image 64 and onthe boundary line 66 d between the right lateral-side image 63 and therear image 64.

Therefore, for example, as illustrated in FIG. 5 , the image processor57 changes the boundary regions so that a boundary line 69 c between theleft lateral-side image 62 and the rear image 64 and a boundary line 69d between the right lateral-side image 63 and the rear image 64 do notoverlap the obstacle 68. Specifically, the boundary line 69 c betweenthe left lateral-side image 62 and the rear image 64 and the boundaryline 69 d between the right lateral-side image 63 and the rear image 64are shifted toward lateral sides, respectively. Accordingly, at the timeof parking the vehicle into the parking space P, the visibility of theobstacle 68 is improved, and the obstacle 68 can be quickly and reliablyrecognized.

Display Control Performed by Control ECU 20

Next, display control performed by the control ECU 20 will be described.

First Embodiment

A first embodiment of the display control performed by the control ECU20 will be described with reference to FIGS. 6 to 8 .

FIG. 6 is a flowchart illustrating display control performed by thecontrol ECU 20 when a parking frame is selected to park the vehicle 10.FIG. 7 is a diagram illustrating an example of a bird’s-eye view imagegenerated using respective pieces of imaging data obtained by the frontcamera 12Fr, the rear camera 12Rr,the left lateral-side camera 12L, andthe right lateral-side camera 12R of the vehicle 10. FIG. 8 is a diagramillustrating an example of a bird’s-eye view image and athree-dimensional image displayed on the touch screen 42 of the vehicle10.

For example, it is assumed that a driver driving the vehicle 10 attemptsto park the vehicle 10 in a parking lot. The control ECU 20 determineswhether there is a request for parking assistance from the driver of thevehicle 10 (step S11). The parking assistance request is output to thecontrol ECU 20 as a parking assistance signal, for example, based on thedriver’s operation on an automatic parking assistance button or aparking auxiliary assistance button in the operation input part 14.

When there is no request for parking assistance in step S11 (step S11:No), the control ECU 20 waits until there is a request for parkingassistance.

When there is a request for parking assistance in step S11 (step S11:Yes), the control ECU 20 causes the image processor 57 to generate abird’s-eye view image and a three-dimensional image based on asynthesized image of respective pieces of imaging data obtained by thefront camera 12Fr, the rear camera 12Rr, the left lateral-side camera12L, and the right lateral-side camera 12R (step S12).

Next, the control ECU 20 causes the image processor 57 to determine,based on a generated image, whether there is a parking frame in theparking lot in which the vehicle 10 can be parked, that is, an availableparking frame in which no vehicle is parked (step S13). Thedetermination as to whether there is an available parking frame may bemade based on the synthesized image of the respective pieces of imagingdata obtained by the front camera 12Fr, the rear camera 12Rr, the leftlateral-side camera 12L, and the right lateral-side camera 12R, or maybe made based on images of the respective pieces of imaging data beforebeing synthesized. Alternatively, the determination may be made based onthe bird’s-eye view image or the three-dimensional image generated basedon the synthesized image.

For example, FIG. 7 illustrates a bird’s-eye view image 70 used fordetermining whether there is an available parking frame. In thebird’s-eye view image 70, a state is displayed where other vehicles Vare parked in three parking frames P1, P2, and P4 among five parkingframes P1 to P5 and no other vehicles V are parked in the two parkingframes P3 and P5. Boundary lines 76 a to 76 d are displayed at boundaryportions of a front image 71, a left lateral-side image 72, a rightlateral-side image 73, and a rear image 74. A vehicle image 77indicating the vehicle 10, which is an own vehicle, is displayed in amask area 75.

In step S13,the control ECU 20 determines, based on the bird’s-eye viewimage 70, that the parking frames P3 and P5 are available parkingframes.

Next, the control ECU 20 causes the image processor 57 to determinewhether the boundary lines 76 a to 76 d overlap the parking frames P3and P5 determined to be available (step S14).

When the boundary lines 76 a to 76 d do not overlap the availableparking frames P3 and P5 in step S14 (step S14: No), the control ECU 20causes the display controller 55 to display the bird’s-eye view imageand the three-dimensional image generated in step S12 on the touchscreen 42 of the vehicle 10 as a parking frame selection screen forparking the vehicle 10 without changing the bird’s-eye view image andthe three-dimensional image (step S16).

When the boundary lines 76 a to 76 d overlap the available parkingframes P3 and P5 in step S14 (step S14: Yes), the control ECU 20 causesthe image processor 57 to change the boundary lines by performing thesynthesis processing of the respective pieces of imaging data again sothat the boundary lines overlapping the parking frames P3 and P5 amongthe boundary lines 76 a to 76 d in the three-dimensional image generatedin step S12 do not overlap the parking frames P3 and P5 (step S15).

Next, the control ECU 20 causes the display controller 55 to display, onthe touch screen 42 and as a parking frame selection screen, thebird’s-eye view image generated in step S12 without change and thethree-dimensional image that is generated in step S15 so that theboundary lines 76 a to 76 d do not overlap the parking frames P3 and P5(step S16).

For example, in the case of the bird’s-eye view image 70 illustrated inFIG. 7 , the boundary line 76 c among the boundary lines 76 a to 76 doverlaps the available parking frame P5. Therefore, the control ECU 20causes the image processor 57 to perform again the synthesis processingon the respective pieces of imaging data so that the boundary line 76 cdoes not overlap the available parking frame P5. and changes to aboundary line 78 c that does not overlap the parking frame P5.

Then, as illustrated in FIG. 8 , among a first display area 42 a and asecond display area 42 b provided in the touch screen 42, the controlECU 20 causes the display controller 55 to display the three-dimensionalimage in which the boundary lines do not overlap the parking frame P5 ina first display area 42 a, and to display the bird’s-eye view image (thebird’s-eye view image 70 in which the boundary line 76 c is not changed)generated in step S12 in the second display area 42 b.

The driver of the vehicle 10 selects a parking frame by touching anavailable parking frame (for example, any one of the parking frames P3and P5) displayed on the touch screen 42. The control ECU 20 performsparking assistance for parking the vehicle 10 in the selected parkingframe by automatic steering.

Second Embodiment

A second embodiment of the display control performed by the control ECU20 will be described with reference to the flowchart illustrated in FIG.9 . In the first embodiment described above, the display control ofchanging a boundary region (boundary line) of only a three-dimensionalimage among a bird’s-eye view image and the three-dimensional imagedisplayed on the touch screen 42 when a predetermined object (parkingframe) is present on the boundary line has been described. In the secondembodiment, display control of changing boundary regions (boundarylines) in both a three-dimensional image and a bird’s-eye view imagewill be described.

The control ECU 20 determines whether there is a request for parkingassistance from the driver of the vehicle 10 (step S21). The parkingassistance request is output to the control ECU 20 based on an operationon an automatic parking assistance button or a parking auxiliaryassistance button as in the first embodiment.

When there is no request for parking assistance in step S21 (step S21:No), the control ECU 20 waits until there is a request for parkingassistance.

When there is a request for parking assistance in step S21 (step S21:Yes), the control ECU 20 causes the image processor 57 to generate abird’s-eye view image and a three-dimensional image based on asynthesized image of respective pieces of imaging data obtained by thefront camera 12Fr, the rear camera 12Rr, the left lateral-side camera12L, and the right lateral-side camera 12R (step S22).

Next, the control ECU 20 causes the image processor 57 to determine,based on a generated image, whether there is a parking frame in theparking lot in which the vehicle 10 can be parked, that is, an availableparking frame in which no vehicle is parked (step S23). As in the firstembodiment, the determination as to whether there is an availableparking frame may be made based on the synthesized image of therespective pieces of imaging data, may be made based on images of therespective pieces of imaging data before being synthesized, or may bemade based on the bird’s-eye view image or the three-dimensional imagegenerated from the synthesized image.

Next, as in the first embodiment, the control ECU 20 determines whetherthe boundary lines 76 a to 76 d overlap the parking frames P3 and P5(see FIG. 7 ) determined to be available (step S24).

When the boundary lines 76 a to 76 d do not overlap the availableparking frames P3 and P5 in step S24 (step S24: No), the control ECU 20causes the display controller 55 to display the bird’s-eye view imageand the three-dimensional image generated in step S22 without change onthe touch screen 42 of the vehicle 10 as a parking frame selectionscreen (step S25).

When the boundary lines 76 a to 76 d overlap the available parkingframes P3 and P5 in step S24 (step S24:Yes), the control ECU 20 causesthe image processor 57 to change the boundary lines by performing thesynthesis processing of the respective pieces of imaging data again sothat firstly boundary lines overlapping the parking frames P3 and P5among the boundary lines 76 a to 76 d in the three-dimensional image,among the bird’s-eye view image and the three-dimensional imagegenerated in step S22, do not overlap the parking frames P3 andP5 (stepS26).

The control ECU 20 displays, on the touch screen 42 and as a parkingframe selection screen, the three-dimensional image that is generatedagain in step S26 so that the boundary lines 76 a to 76 d do not overlapthe parking frames P3 and P5 (step S27).

Next, the control ECU 20 changes the boundary lines by performing thesynthesis processing of the respective pieces of imaging data again sothat the boundary lines overlapping the parking frames P3 and P5 amongthe boundary lines 76 a to 76 d in the bird’s-eye view image, among thebird’s-eye view image and the three-dimensional image generated in stepS22, do not overlap the parking frames P3 and P5 (step S28).

The control ECU 20 causes the display controller 55 to display, on thetouch screen 42 and as a parking frame selection screen, the bird’s-eyeview image that is generated again in step S28 so that the boundarylines 76 a to 76 d do not overlap the parking frames P3 and P5 (stepS29).

In this case, in FIG. 8 , the bird’s-eye view image displayed in thesecond display area 42 b of the touch screen 42 is also displayed as animage in which the boundary lines do not overlap the parking frame P5,similarly to the three-dimensional image displayed in the first displayarea 42 a.

Third Embodiment

A third embodiment of the display control performed by the control ECU20 will be described with reference to FIGS. 10 to 12 .

FIG. 10 is a diagram illustrating an example of a bird’s-eye view imagegenerated using respective pieces of imaging data obtained by the frontcamera 12Fr, the rear camera 12Rr, the left lateral-side camera 12L, andthe right lateral-side camera 12R of the vehicle 10. FIG. 11 is adiagram illustrating an example of a three-dimensional image generatedusing the same respective pieces of imaging data obtained by the cameras12Fr, 12Rr, 12L, and 12R. FIG. 12 is a diagram illustrating an exampleof the bird’s-eye view image and the three-dimensional image displayedon the touch screen 42 of the vehicle 10.

The third embodiment illustrates display control performed by thecontrol ECU 20 in a case where the vehicle 10 is back-parked in thepredetermined parking space P. As illustrated in FIGS. 10 and 11 , aparking frame line 88 serving as a parking target position of thevehicle 10 is provided in the parking space P. Therefore, at the time ofthe back parking, an image of the parking frame line 88 with goodvisibility is required in order to enable accurate recognition of apositional relationship between the vehicle 10 (a vehicle image 87) andthe parking frame line 88.

Therefore, for example, as shown in a bird’s-eye view image 80A of FIG.10 , in a case where it is determined that the parking frame line 88 isdisplayed overlapping a boundary line 86 c between the left lateral-sideimage 82 and the rear image 84 and a boundary line 86 d between theright lateral-side image 83 and the rear image 84 when the vehicle 10goes back, the control ECU 20 causes the image processor 57 to performthe synthesis processing of the respective pieces of imaging data againso that the parking frame line 88 and the boundary lines 86 c and 86 ddo not overlap each other.

Then, the boundary lines 86 c and 86 d are changed to the boundary linesshifted to the lateral side not overlapping the parking frame line 88,like boundary lines 89 c and 89 d shown in the bird’s-eye view image 80Aof FIG. 10 and a three-dimensional image 80B of FIG. 11 . With respectto the change of the boundary lines, the boundary lines in only thethree-dimensional image 80B may be changed as in the first embodiment,or the boundary lines in the bird’s-eye view image 80A may be furtherchanged after the boundary lines in the three-dimensional image 80B ischanged as in the second embodiment.

Accordingly, the touch screen 42 of the vehicle 10 displays images amongwhich at least the three-dimensional image 80B does not have a boundaryline overlapping the parking frame line 88, that is, thethree-dimensional image 80B in which the parking frame line 88 havinggood visibility is displayed is displayed on the touch screen 42. In thefirst display area 42 a and the second display area 42 b of the touchscreen 42 illustrated in FIG. 12 , the three-dimensional image 80B andthe bird’s-eye view image 80A in which no boundary line overlaps theparking frame line 88 are displayed, respectively.

As described above, when the object presence/absence determination unit56 determines that a predetermined object is present on the boundaryline, the control ECU 20 causes the image processor 57 to preferentiallychange the boundary line in the three-dimensional image among thedisplayed bird’s-eye view image and the three-dimensional image.

Note that preferentially changing the boundary line in thethree-dimensional image among the bird’s-eye view image and thethree-dimensional image means preferentially changing the boundary linein the three-dimensional image unless there is any other factor topreferentially change the boundary line in the bird’s-eye view image,such as the user designating to change the boundary line in thebird’s-eye view image or the user frequently referring to the bird’s-eyeview image rather than the three-dimensional image.

Accordingly, when a predetermined object is present on the boundaryline, the boundary line in the three-dimensional image, with which asurrounding situation is more easily recognized by the driver of thevehicle 10 than with the bird’s-eye view image, is changedpreferentially (namely, on a priority basis), and thus it is possiblefor the driver to quickly recognize the predetermined object. Therefore,for example, it is possible to accurately check whether the vehicle 10collides with an obstacle in the surroundings while the vehicle 10 isentering a narrow parking space or coming out from a narrow parkingspace. In addition, while the vehicle 10 is entering the narrow parkingspace, it is easy to check whether there is a space for allowing theoccupant of the vehicle 10 to easily get off the vehicle 10 after thevehicle 10 is stopped. In addition, while the vehicle 10 is stopping, itis easy to check whether there is an obstacle that the occupant of thevehicle 10 comes into contact with at the time of getting off thevehicle 10.

When the object presence/absence determination unit 56 determines that apredetermined object is present on the boundary line, the control ECU 20causes the image processor 57 to change the boundary line in only thethree-dimensional image among the bird’s-eye view image and thethree-dimensional image. For this reason, at least the boundary line inthe three-dimensional image with which the surrounding situation iseasily recognized is changed, and thus it is possible to rapidlyrecognize the predetermined object.

When the object presence/absence determination unit 56 determines that apredetermined object is present on the boundary line, the control ECU 20causes the image processor 57 to change the boundary line in thethree-dimensional image and displays the changed three-dimensional imageon the touch screen 42, and then changes the boundary line in thebird’s-eye view image and displays the changed bird’s-eye view image onthe touch screen 42. Accordingly, it is possible to quickly recognizethe predetermined object through the three-dimensional image, and it ispossible to check the object in images having good visibility throughthe bird’s eye view image as well, and thus the convenience is improved.

Although a case is described in the embodiment described above wherewhen it is determined that a predetermined object is displayedoverlapping the boundary line, the boundary line in thethree-dimensional image among the bird’s-eye view image and thethree-dimensional image is always preferentially changed, the presentdisclosure is not limited thereto. For example, the control ECU 20 maypreferentially change the boundary line in one of the bird’s-eye viewimage and the three-dimensional image based on information related tothe user of the vehicle 10 (for example, the driver of the vehicle 10).

The information related to the user is, for example, a setting by theuser. That is, when the user of the vehicle 10 (for example, the driverof the vehicle 10) sets that the boundary line in the bird’s-eye viewimage among the bird’s-eye view image and the three-dimensional imageshould be preferentially changed, the control ECU 20 may preferentiallychange the boundary line in the bird’s-eye view image among thebird’s-eye view image and the three-dimensional image. Accordingly, itis possible to improve the usability of the present function in thevehicle 10.

Alternatively, the information related to the user may be historyinformation of the user referring to each of the bird’s-eye view imageand the three-dimensional image in the past. For example, the controlECU 20 may determine which image of the bird’s-eye view image and thethree-dimensional image the driver of the vehicle 10 more frequentlyrefers to, based on the history information of the user referring toeach of the bird’s-eye view image and the three-dimensional image in thepast, and may preferentially change the boundary line in the image thatis more frequently referred to. Accordingly, it is possible to improvethe usability of the present function in the vehicle 10.

The history information of the user referring to each of the bird’s-eyeview image and the three-dimensional image in the past is obtained, forexample, based on a detection result by a line-of-sight sensor that isprovided in the vehicle 10 and that detects a line of sight of thedriver of the vehicle 10. In addition, in a case where any one of thebird’s-eye view image and the three-dimensional image can be displayedon the touch screen 42 according to an operation of the driver, thehistory information of the user referring to each of the bird’s-eye viewimage and the three-dimensional image in the past may be obtained basedon a switching history of display of the bird’s-eye view image and thethree-dimensional image that is operated by the driver.

Although the three-dimensional image and the bird’s-eye view imagedisplayed at the time of selecting a parking frame or at the time ofparking the vehicle 10 have been described, the present disclosure isnot limited thereto, and can be applied to the three-dimensional imageand the bird’s-eye view image displayed at the time of starting to movethe vehicle 10.

Although the embodiment of the present disclosure has been describedabove, the present disclosure is not limited to the above-describedembodiment, and modifications, improvements, and the like can be made asappropriate.

For example, although a case where the control ECU 20 displays thebird’s-eye view image and the three-dimensional image on the touchscreen 42 of the vehicle 10 has been described in the above-describedembodiment, the present disclosure is not limited thereto. For example,the control ECU 20 may display the bird’s-eye view image and thethree-dimensional image on a display screen of an information terminal(for example, a smartphone) possessed by the occupant of the vehicle 10via the communication unit 24.

Although an example in which the moving body is a vehicle is describedin the above-described embodiment, the present disclosure is not limitedthereto. The concept of the present disclosure can be applied not onlyto a vehicle but also to a robot, a boat, an aircraft, and the like thatare provided with a driving source and movable by power of the drivingsource.

The control method described in the above embodiment can be implementedby executing a control program prepared in advance on a computer. Thecontrol program is recorded in a non-transitory computer-readablestorage medium and is executed by being read from the storage medium.The control program may be provided in a form stored in a non-transitorystorage medium such as a flash memory, or may be provided via a networksuch as the Internet. The computer that executes the control program maybe provided in a control device, may be provided in an electronic devicesuch as a smartphone, a tablet terminal, or a personal computer capableof communicating with the control device, or may be provided in a serverdevice capable of communicating with the control device and theelectronic device.

In the present specification, at least the following matters aredescribed. Although the corresponding components or the like in theabove-described embodiment are shown in parentheses, the presentdisclosure is not limited thereto.

(1) A control device, including: an image processor (image processor 57)that generates a bird’s-eye view image and a three-dimensional imagethat show a moving body (vehicle 10) and surroundings of the movingbody, based on respective pieces of imaging data obtained by a pluralityof imaging devices (front camera 12Fr, rear camera 12Rr,leftlateral-side camera 12L, and right lateral-side camera 12R) of themoving body;

-   a display controller (display controller 55) that causes a display    device (touch screen 42) to display the bird’s-eye view image and    the three-dimensional image generated by the image processor; and-   a determination unit (object presence/absence determination unit 56)    that determines whether a predetermined object (obstacle 68, parking    frame line 88) is present in a boundary region between the    respective pieces of imaging data in the bird’s-eye view image and    the three-dimensional image.-   in which when the determination unit determines that the    predetermined object is present in the boundary region, the image    processor preferentially changes the boundary region in the    three-dimensional image among the bird’s-eye view image and the    three-dimensional image to be displayed.

According to (1), when the predetermined object is present in theboundary region, the boundary region in the three-dimensional image withwhich a surrounding situation is more easily recognized by a user ispreferentially changed, so that the predetermined object can be quicklyrecognized by a driver.

(2) The control device according to (1),

in which when the determination unit determines that the predeterminedobject is present in the boundary region, the image processor changesthe boundary region in only the three-dimensional image among thebird’s-eye view image and the three-dimensional image.

According to (2), since at least the boundary region in thethree-dimensional image with which the surrounding situation is easilyrecognized is changed, it is possible to recognize the predeterminedobject quickly.

(3) The control device according to (1),

in which when the determination unit determines that the predeterminedobject is present in the boundary region, the image processor changesthe boundary region in the three-dimensional image and output thechanged three-dimensional image to the display controller, and thenchanges the boundary region in the bird’s-eye view image and outputs thechanged bird’s-eye view image to the display controller.

According to (3), it is possible to quickly recognize the predeterminedobject through the three-dimensional image, and it is possible to checkthe predetermined object in images having good visibility through thebird’s-eye view image, and thus the convenience is improved.

(4) The control device according to any one of (1) to (3),

in which when the determination unit determines that the predeterminedobject is present in the boundary region, the image processorpreferentially changes the boundary region in one of the bird’s-eye viewimage and the three-dimensional image based on information related to auser of the moving body.

According to (4), it is possible to improve usability by preferentiallychanging the boundary region in an image corresponding to theinformation related to the user among the bird’s-eye view image and thethree-dimensional image.

(5) The control device according to (4),

in which the information related to a user of the moving body includesinformation of history of the user referring to each of the bird’s-eyeview image and the three-dimensional image in the past.

According to (5), it is possible to improve the usability bypreferentially changing the boundary region in an image that the userrefers more frequently among the bird’s-eye view image and thethree-dimensional image.

(6) A control method to be executed by a processor, the processor beingconfigured to generate a bird’s-eye view image and a three-dimensionalimage that show a moving body and surroundings of the moving body basedon respective pieces of imaging data obtained by a plurality of imagingdevices of the moving body, and display the generated bird’s-eye viewimage and the generated three-dimensional image on a display device, thecontrol method including:

-   the processor determining whether a predetermined object is present    in a boundary region between the respective pieces of imaging data    in the bird’s-eye view image and the three-dimensional image; and-   when it is determined that the predetermined object is present in    the boundary region, the processor preferentially changing the    boundary region in the three-dimensional image among the bird’s-eye    view image and the three-dimensional image to be displayed.

According to (6), when the predetermined object is present in theboundary region, the boundary region in the three-dimensional image withwhich a surrounding situation is more easily recognized by a user ispreferentially changed, so that the predetermined object can be quicklyrecognized by a driver.

(7) A control program for causing a processor to perform processing, theprocessor being configured to generate a bird’s-eye view image and athree-dimensional image that show a moving body and surroundings of themoving body based on respective pieces of imaging data obtained by aplurality of imaging devices of the moving body, and to display thegenerated bird’s-eye view image and the generated three-dimensionalimage on a display device, the processing including:

-   determining whether a predetermined object is present in a boundary    region between the respective pieces of imaging data in the    bird’s-eye view image and the three-dimensional image; and-   when it is determined that the predetermined object is present in    the boundary region, preferentially changing the boundary region in    the three-dimensional image among the bird’s-eye view image and the    three-dimensional image to be displayed.

According to (7), when the predetermined object is present in theboundary region, the boundary region in the three-dimensional image withwhich a surrounding situation is more easily recognized by a user ispreferentially changed, so that the predetermined object can be quicklyrecognized by a driver.

1. A control device, comprising: circuitry configured to: generate abird’s-eye view image and a three-dimensional image that show a movingbody and surroundings of the moving body, based on respective pieces ofimaging data obtained by a plurality of imaging devices of the movingbody; cause a display device to display the generated bird’s-eye viewimage and the generated three-dimensional image; and determine whether apredetermined object is present in a boundary region between therespective pieces of imaging data in the bird’s-eye view image and thethree-dimensional image, wherein upon determining that the predeterminedobject is present in the boundary region, the circuitry is configured topreferentially change the boundary region in the three-dimensional imageamong the displayed bird’s-eye view image and the displayedthree-dimensional image.
 2. The control device according to claim 1,wherein upon determining that the predetermined object is present in theboundary region, the circuitry is configured to change the boundaryregion in only the displayed three-dimensional image among thebird’s-eye view image and three-dimensional image.
 3. The control deviceaccording to claim 1, wherein upon determining that the predeterminedobject is present in the boundary region, the circuitry is configured tochange the boundary region in the three-dimensional image and output todisplay the changed three-dimensional image, and change the boundaryregion in the bird’s-eye view image and output to display the changedbird’s-eye view image.
 4. The control device according to claim 1,wherein upon determining that the predetermined object is present in theboundary region, the circuitry is configured to change the boundaryregion in one of the bird’s-eye view image and the three-dimensionalimage based on information related to a user of the moving body.
 5. Thecontrol device according to claim 4, wherein the information related tothe user of the moving body includes information of history of the userreferring to each of the bird’s-eye view image and the three-dimensionalimage in the past.
 6. A control method executed by a processor, whereinthe processor is configured to generate a bird’s-eye view image and athree-dimensional image that show a moving body and surroundings of themoving body based on respective pieces of imaging data obtained by aplurality of imaging devices of the moving body, and display thegenerated bird’s-eye view image and the generated three-dimensionalimage on a display device, and the control method comprises: theprocessor determining whether a predetermined object is present in aboundary region between the respective pieces of imaging data in thebird’s-eye view image and the three-dimensional image; and upondetermining that the predetermined object is present in the boundaryregion, the processor preferentially changing the boundary region in thethree-dimensional image among the displayed bird’s-eye view image andthe displayed three-dimensional image.
 7. A non-transitorycomputer-readable storage medium storing a control program for causing aprocessor to perform processing, wherein the processor is configured togenerate a bird’s-eye view image and a three-dimensional image that showa moving body and surroundings of the moving body based on respectivepieces of imaging data obtained by a plurality of imaging devices of themoving body, and to display the generated bird’s-eye view image and thegenerated three-dimensional image on a display device, and theprocessing comprises: determining whether a predetermined object ispresent in a boundary region between the respective pieces of imagingdata in the bird’s-eye view image and the three-dimensional image; andwhen it is determined that the predetermined object is present in theboundary region, preferentially changing the boundary region in thethree-dimensional image among the displayed bird’s-eye view image andthe displayed three-dimensional image.